The following U.S. Patents have been found in a U.S. Patent Search and are believed to be generally relevant to the field of the invention:
______________________________________ 5,022,085 6/91 Cok 5,155,586 10/92 Levy et al. 5,185,808 2/93 Cok 5,231,385 7/93 Gengler et al. 5,251,022 10/93 Kitamura 5,325,449 6/94 Burt et al. 5,488,674 1/96 Burt et al. 5,630,037 5/97 Schindler 5,649,032 7/97 Burt et al. ______________________________________
Digital photographs with extended fields of view are conventionally created using an image processing technique known as "stitching", wherein a user takes multiple photographic images using a camera and rotates the camera between images, so as to produce overlapping photos spanning an entire vista. A "stitcher" then combines the various images into a single large "omniview" image with an extended field of view, up to a full 360.degree.. The stitching process itself involves various steps, such as correction of the images for camera angle and tilt, image alignment between successive images to account for overlap, and texture mapping. When performed accurately and properly viewed with appropriate perspective correction, the resulting large image gives the viewer the experience of being immersed within a vista.
Although certain professional lenses, such as fisheye lenses, are capable of photographing an entire hemispherical view as a single image, use of a stitcher enables a non-professional photographer to use a standard lens, and yet capture the same extended wide angle view. The computational processing within the stitcher effectively carries out necessary three-dimensional geometrical transformations to account for the non-linear distortion in a wide angle view. Specifically, it projects the various linear images onto the surface of a sphere or cylinder or cube--depending on a desired environment map.
One commonly encountered source of artifacts in stitching is the variable lighting of individual images. As the camera rotates, for example, one image can be taken into the sun, and thus deviate from the other images with respect to lighting. When this image is stitched with the others, it stands out awkwardly.
Current state of the art stitchers use a process known as "feathering" to merge together a left image (hereinafter designated as "image A") and an adjacent right image (hereinafter designated "image B"). Typical implementations of feathering operate after the images are aligned and the overlap region has been identified. They then feather the two images within the overlap region, by weighted combining using weights which vary from a value above 50% (e.g. 100%) of image A and a value below 50% (e.g. 0%) image B at the left side of the overlap region, to a value below 50% (e.g. 0%) of image A and a value above 50% (e.g. 100%) of image B at the right side of the overlap region. However, if the lighting conditions of the two images are substantially at variance and the overlap region is narrow, this technique of feathering does not work well, and typically produces noticeable bands at the overlap region. This is to be expected, since feathering does not affect pixel color values outside of the overlapping region.
Another common artifact, referred to as "ghosting," occurs when two images being blended together are slightly misaligned. Suppose, for example, that the overlap region contains a bright square of dimensions 1".times.1". If the two images are mis-aligned horizontally by 0.1", then feathering will produce an elongated square (i.e. a rectangle) of dimensions 1.1".times.1". Moreover the leftmost and rightmost strips of the rectangle, of dimensions 0.1".times.1", will be less bright than the rest of the rectangle. The appearance of these less bright strips at the left and right ends of the rectangle gives the effect of ghosting, whereby only a partial remnant of the square is present. It should be emphasized that alignment is practically never perfect, and as such, there is always some form of ghosting present.
Another source of ghosting occurs when an object is present in one of the images and absent in the other. For example, a person might appear in image A and be absent from image B, having moved out of sight between the times the two images were photographed.